Add a new section for transitioning indices to data streams #2216
Conversation
|
I'm also wondering whether this tutorial is related to the goal we're trying to achieve in the Migrate from Indices to DataStreams knowledgebase article. |
Also, applying some editiorial changes to blend in with the structure of the page
…ios described Hopefully this helps someone decide which procedure to follow.
manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md
Outdated
Show resolved
Hide resolved
manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md
Outdated
Show resolved
Hide resolved
manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md
Outdated
Show resolved
Hide resolved
manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md
Outdated
Show resolved
Hide resolved
| ### Roll over the reindexed data stream [manage-general-content-with-data-streams-roll-over] | ||
|
|
||
| Use the [_rollover API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-rollover) to create a new write index for the stream. This ensures that the lifecycle policy and ingest pipeline you've created will apply to any new documents that you index. |
There was a problem hiding this comment.
If we have already created an index and we have (maybe) performed a reindex, everything should be in place. What's the point of a rollover API call?
The user just needs to send data to the data stream, the rollover API should be used manually when you want an exceptional / extra rollover for any reason. You can do it, definitely, it's harmless, but you will cause a rollover regardless if the backend index really needs to be rolled over based on the ILM settings.
There was a problem hiding this comment.
Thanks, my plan is to check with Zoia on the intention of this step and then remove this guidance if the step is just checking that indices get rolled over.
I can mention that info in the procedure description, as an optional step and remove this section as there's no need to elaborate on it.
There was a problem hiding this comment.
Hi @eedugon -- I've confirmed with Zoia that this step is redundant and have removed it. I've also addressed all of your other suggestions, thanks for all your help.
If you'd like to give this PR another glance over before I merge it, please go ahead.
|
Hi @kilfoyle, I've implemented most of the changes that Edu suggested (one item is pending, I'm checking with the author about the goal of the manual rollover step), so depending on that, I might need to remove that step. Other than that, it should be ready for another peer review, whenever you get the chance. Please and thank you! 😃 P.S. The build seems to be broken now, but you should be able to use the preview regardless. |
manage-data/lifecycle/index-lifecycle-management/tutorial-automate-rollover.md
Outdated
Show resolved
Hide resolved
I've confirmed with the knowledgebase article author that the manual rollover step is redundant, therefore I'm removing it.
The goal of this PR is to include the content from the Migrate from Indices to DataStreams knowledgebase article into our ILM-related documentation.
Initially, the suggestion was to include this new content as a new section on the Manage existing indices page. Upon further review of existing content, the Tutorial: Automate rollover seems like a more appropriate home for this content as it already includes two other equivalent use cases:
The reason I think these use cases are equivalent is because they're trying to use ILM policies to migrate from periodic indices to a more automated way to manage rollover and replace the need to schedule or script index creation (one option migrates from indices to data streams and the other one migrates to using aliases in order to manage their backing indices). The new content adds this use case that's equivalent in scope:
Fixes #1571